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POSITION MEASUREMENT METHOD, AN APPARATUS, A COMPUTER 
PROGRAM AND A METHOD FOR GENERATING CALIBRATION INFORMATION 

CROSS-REFERENCE TO RELATED APPLICATIONS 

[000.1] The present invention claims priority under 35 USC 119 based on Japanese 

patent application No. 2003-103498, filed on April 07, 2003, as well as on Japanese patent 
application No. 2003-1 10466. filed on April 15. 2003. The subject matter of each of these 
priority documents is incorporated by reference herein. 

BACKGROUND OF THE INVENTION 

[001] Field of the Invention 

The present invention relates to a technique which provides measurement of the 
position of an object from an image captured by a camera unit. 

[002] 2. Discussion of Background Art 

Three-dimensional measurement with images has recently been prevailing as a result 
of development of processing for the image of a CCD camera or a computer. There are 
examples such as a technique which measures the position of an object with plural cameras, 
and another one which uses a combination of a camera and a floodlight. The former calculates 
a distance from the camera to the object with tri angulation incorporating a position of each 
camera and a pixel representative of the object in an image captured by each camera. On the 
other hand, the latter calculates it with triangulation based on a relative position between the 



camera and floodlight, and a position of a beam of collimated light in an image captured by 
the camera. Japanese Published Patent Application 9-287927 reports related arts associated 
with the latter technique. In this connection, these techniques are based on the assumption that 
a lens system of the camera can be modeled by a pin-hole camera. As shown in FIG.l, a model 
of pin-hole camera is characterized in that only a beam of light coming through a pin-hole H 
reaches a plane of image, so that a three-dimensional space (x, y, z) is translated into a 
two-dimensional space (u, v). The model is thus generated on condition that the incident beam 
of light passes through the pin-hole, focusing an image in the camera. 

[003] However, it is known that the image captured by the camera with the lens system has 

intrinsic non-linear distortion, and the further it departs from the center, the larger the 
distortion will be. In this way, it is not possible to measure the accurate position of an object 
from the image having this type of distortion. A technique for improving the accuracy with 
compensation for the captured image has been proposed. Japanese Published Patent 
Application 5-274426 reports related arts. 

[004] The related arts described above, which make the compensation for the image based on 

the phenomenon that the peripheral distortion is larger, does not provide fundamental 
compensation. For example, selecting a matrix of feature points in the center of an image 
where the distortion is regarded as relatively limited, the technique disclosed by Japanese 
Published Patent Application 5-274426 calculates a reference point applied to the whole 
feature points of the image. Because the center is not free from the distortion, the reference 
point is accordingly calculated from the image having the distortion. In this way, although it 
may be possible for this technique to improve the accuracy for measurement of a position to 
some extent, there still remains a problem that the sufficiently accurate position measurement 
may not necessarily be implemented. 
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[005] The conventional compensation for the distortion of lens system depends on a distance 

between the camera and object. If the distance is known, it may be possible to make 
compensation with a function of compensation for the distortion of image, which varies 
non-linearly according to the distance. However, it has not yet solved the problem because 
actually the distance is unknown before the measurement of a position is completed. 

SUMMARY OF THE INVENTION 

[006] The present invention is directed to overcoming of the problem [[above]] described 

above . It seeks to provide a position measurement method, which fundamentally compensates 
the distortion of an image captured by a camera unit with a lens system and measures the 
accurate position of an object accordingly, as well as an apparatus, a computer program and a 
method for generating calibration information to be used for the position measurement method 
and the apparatus. 

[007] According to an aspect of the present invention, a method is provided, which is applied 

to measurement for a position of an object according to an image of the object captured by a 
camera unit. The method includes the following steps: calculating a discrepancy of an incident 
beam of light penetrating a lens system of the camera unit relative to an optical center of the 
lens system, and compensating the position of the object according to the discrepancy. 

[008] The method described above, which calculates the discrepancy of the incident beam of 

light penetrating the lens system with regard to its optical center, allows recognition of the 
displacement of the pixel which the camera unit finds representative of the position of the 
object. In this way, it is possible to detect the accurate position of the object by making 
compensation according to the displacement of the pixel representative of the position of 
object with regard to the optical center. In this connection, the optical center is meant to 



represent the center of the lens system, corresponding to the position of a pin hole in case of a 
pin-hole camera model. 

[009] According to another aspect of the present invention, a method is provided, which is 

applied to measurement for a position of an object with a combination of an image of the 
object captured by a camera unit and calibration information. The calibration information is 
prepared in advance in such a manner that a position of a measurement pixel of the image is 
correlated with a direction of an incident beam of light and a displacement from a reference 
point to the incident beam. The method includes the following steps: (a) incorporating the 
image without any mask restricting an incident beam of light , (b) detecting a position of a 
pixel representative of the object in the image incorporated at step (a); and (c) calculating the 
position of the object according to the direction and the displacement of the incident beam, 
which are obtained from the calibration information with reference to the position of the pixel 
detected at step (b). 

[010] The method described above, which recognizes the discrepancy of the incident beam of 

light that impinges on the pixel representative of the object before detection, can detect the 
accurate position of the object by making compensation according to the discrepancy. 

[Oil] According to still another aspect of the present invention, an apparatus is provided, 

which measures a position of an object according to an image of the object captured by a 
camera unit. The apparatus includes an image input means, a pixel position detection means, a 
storage means for storing calibration information and a position calculation means for 
calculating the position of the object according to the direction and the displacement of the 
incident beam of light. 

[012] The apparatus described above has the image input means which incorporates the 

image of the object taken by the camera unit and the pixel position detection means which 



detects the position of the pixel representative of the object. The apparatus also has the 
position calculation means which calculates the position of the object according to the 
direction and the displacement of the incident beam of light that are derived from the 
calibration information. 

[013] According to yet another aspect of the present invention, an apparatus is provided, in 

which a camera unit includes cameras in sets of at least two so as to take a plurality of images 
and a storage means stores the calibration information for each camera. 

[014] The apparatus described above can recognize the discrepancy of the incident beam of 

light impinging on the pixel representative of the object of each image before it detects the 
position of the object according to the plurality of images because the storage means stores the 
calibration information for each camera. In this way, the apparatus can detect the accurate 
position of the object by compensation according to the discrepancy. 

[015] According to a further aspect of the present invention, an apparatus is provided, in 

which a pixel position detection means detects a position of a pixel representative of an object 
that has a marker identifying its typical spot. 

[016] The apparatus described above, in which the pixel position detection means detects the 

position of the pixel according to the marker, can concentrate its effort of detection on the 
marked spot of the object. The apparatus can detect the accurate position of the object by 
compensation according to the discrepancy because it recognizes the discrepancy of the 
incident beam of light impinging on the pixel representative of the marker before it detects the 
position of the marker in the image. In this connection, it may alternatively be possible to 
select a spot such as an edge of an object instead of introducing a marker. 

[017] According to a still further aspect of the present invention, an apparatus is provided, 

which further includes a floodlight that projects a collimated beam of light on the object. A 



camera unit captures an image of the object illuminated by the collimated beam, and the 
apparatus measures the position of the object according to spatial relationship between a 
position of the floodlight and an optical center of the camera unit as well as a position of the 
collimated beam on the image. 

[018] The floodlight of the apparatus described above projects the collimated beam of light 

on the object. The collimated beam may include a spot light beam, a slit light beam in the form 
of bands and a pattern light beam having a given pattern, as long as it projects a beam of light 
in a not divergent form. It may also be possible to select a combination of them. 

[019] The image input means incorporates the image captured by the camera unit and the 

pixel position detection means detects the position of the pixel which is illuminated by the 
collimated beam of light in the image. Furthermore, accessing to the calibration information, 
the position calculation means obtains the direction and displacement of the incident beam of 
light representative of the position of the pixel, so that it calculates the position of illumination, 
namely the position of the object, according to the direction and displacement. 

[020] In this way, the apparatus of the present invention can measure the position of the 

object according to the accurate direction of the incident beam of light penetrating the lens 
system. 

[021] According to a yet further aspect of the present invention, a computer program for a 

computer used for an apparatus is provided, which generates calibration information 
correlating a position of a measurement pixel of an image captured by a camera unit with a 
direction of an incident beam of light and a displacement from a reference point to the incident 
beam of light, and measures a position of an object according to an image of the object 
captured by the camera unit and the calibration information. The computer program is 
executed by the computer in a process comprising: (a) incorporating the image of the object, 



(b) detecting a position of a pixel representative of the object in the image incorporated at 
process (a), and (c) calculating the position of the object according to the direction and the 
displacement of the incident beam of light, which are derived from the calibration information 
with reference to the position of the pixel detected at process (b). 

[022] The computer program described above is executed by the computer so that the image 

input means can incorporate the image captured by the camera unit and the pixel position 
detection means can detect the position of the pixel representative of the object in the image. 
In this way, the position calculation means can obtain the direction and the displacement of the 
incident beam of light according to the position of the pixel and the calibration information, 
thereby calculating the position of the object. 

[023] According to another aspect of the present invention, a method for generating 

calibration information is provided, which includes the following steps: projecting a beam of 
light on individual pixels of a camera image; according to the incident beam of light for each 
pixel, calculating a displacement from a reference point to the incident beam of light and 
generating the calibration information by correlating a direction and the displacement of the 
incident beam of light with a position of each pixel. 

[024] The method described above, in which a beam of light is imposed on each pixel of the 

image, specifies the direction of the incident beam of light for the pixel. In this connection, the 
direction of the beam of light may be fixed if measurement is conducted for at least two 
locations of the light source. And the method can provide calculation of a displacement from 
the reference point to the beam of light, thereby allowing generation of the calibration data 
correlating the direction and displacement with the position of the pixel. The data has the 
nature of quantitatively calibrated characteristics of the camera. 

[025] According to still another aspect of the present invention, a method for generating 



calibration information is provided, which includes the following steps: adjusting a first 
direction of a camera unit and measuring a first relative position of the light source relative to 
the camera unit, similarly adjusting a second direction of the camera unit and measuring a 
second relative position of the light source relative to the camera unit, repeating determination 
of an incident beam of light impinging on the measurement pixel according to the first and 
second relative positions for predetermined measurement pixels, calculating a displacement 
from a reference point to the incident beam of light for each of the measurement pixels, and 
generating the calibration information which correlates a direction and the displacement of the 
incident beam of light with each of the measurement pixels. 

[026] In the method described above, adjustment of the first direction of the camera unit is 

made by controlling its pan and tilt so that the first peak intensity of light emitted by the light 
source falls in the measurement pixel captured by the camera unit. In this way, the first 
relative position of the light source relative to the camera unit is measured. The relative 
position measured as described above serves as a first position, based on which the incident 
beam of light impinging on the measurement pixel is determined. 

[027] Similarly, the second relative position of the light source is measured and serves as a 

second position. 

[028] In this way, these two relative positions make it possible to determine the incident 

beam of light impinging on the pixel. 

[029] In the method, a step for determining an incident beam of light is repeated according to 

the number of measurement pixels, and the displacement from the reference point to the 
incident beam is calculated. In this way, the calibration data is generated, which correlates the 
direction and displacement of the incident beam of light with the position of each 
measurement pixel. This leads to quantification of the characteristics of the camera unit. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



[03 1 ] FIG. 1 is a schematic diagram illustrating a pin-hole camera model. 

[032] FIG.2 is a schematic diagram for describing distortion on an image captured by a 

camera. 

[033] FIG. 3 is a schematic diagram for describing the calibration data. 

[034] FIGS.4A and 4B are schematic diagrams conceptually illustrating how to generate the 

calibration information according to the present invention. 
[035] FIG. 5 is a perspective view showing an overall apparatus for generating data for 

calibration information. 

[036] FIG.6 is a flow chart showing steps for generating data for calibration by a calibration 

information generator. 

[037] FIG. 7 is a diagram illustrating a position of light source relative to a camera when the 

camera is panned and tilted. 
[038] FIG.8 is a flow chart showing the steps of a calibration information generator for 

determining a direction of incident beam of light. 
[039] FIG. 9 is a diagram illustrating how to search for a peak position of lightness. 

[040] FIG. 1 0 is a block diagram illustrating the structure of a position measurement 

apparatus according to an embodiment of the present invention. 
[041] FIG.l 1 is a schematic diagram illustrating the contents of a calibration table. 

[042] FIG. 12 is a flow chart illustrating the operation of a position measurement apparatus 

according to the present invention. 
[043] FIG. 13 is a schematic diagram describing a method for calculating the three 



dimensional position of an object. 
[044] FIG. 14 is a block diagram illustrating the structure of a position measurement 

apparatus according to another embodiment the present invention. 
[045] FIG. 1 5 is a block diagram for a laser unit according to the present invention. 

[046] FIG. 1 6 is a perspective view illustrating illuminated laser beams. 

[047] FIG. 17 is a diagram showing a coordinate system and the definition for angles which 

are used in description of calculation method for a position illuminated by a laser beam. 
[048] FIG. 18 is a flow chart showing the steps for operating a position measurement 

apparatus according to the present invention. 

DESCRIPTION OF THE PRESENT EMBODIMENTS 

[049] Embodiments of the present invention are now described in detail. First, description is 

given to characteristics of a camera which intrinsically conflict with an idealized assumption 
of pin-hole camera that incident beams of light cross at a point. This leads to one of the causes 
for distortion of an image captured by the camera with a lens system. Subsequently, 
description is given to calibration information (calibration table) of quantified characteristics 
of the camera, and to a method for generating the calibration information by conducting 
measurement for individual pixels captured by the camera. And description is given to an 
apparatus for detecting a position of an object, which can remove distortion of the image. 

Non-pin-hole Characteristic of a Camera 
[050] Referring to FIG.2, description is given to a cause for creating distortion on an image 

captured by a camera with a lens system. FIG.2 is a schematic diagram of the camera with the 
lens system. For convenience sake, it is assumed that a lens system of a camera unit C is 
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modeled with a plate glass G and a pin hole H is formed by a diaphragm F. An incident beam 
of light rl, which perpendicularly penetrates the plate glass G, forms an image on a pixel Rl 
of an image plane I. Other incident beams of light r2 and r3, which diagonally penetrate the 
plate glass G, form images on pixels R2 and R3 of the image plane I, respectively. 

[051] It is known that the camera unit C cannot be qualified as a pin-hole camera because 

lines r22 and r33, which are extensions of the incident beams of light r2 and r3 before 
penetrating the plate glass G, do not intersect with the incident beam of light rl at one point. In 
this way, the incident beam of light r3, which is offset by a distance D from an incident beam 
of light rr, forms the image on the pixel R3. 

[052] As described above, the characteristics of a pin-hole camera do not hold in a case of 

camera, which forms an image with incident beams of light penetrating a lens system (in this 
case the plate glass G). A camera with a lens system is hereinafter referred to as "non-pin-hole 
camera". 

Calibration Data 

[053] Referring to FIG.3, description is given to calibration data including quantified 

characteristics of a non-pin-hole camera. FIG.3 is a schematic diagram illustrating a lens 
system for description of the calibration data. As shown in FIG.3, an incident beam of light R 
penetrating a lens LS is defined by two points. It is understood that if beams of light 
originating from first and second positions PI and P2 of light source form images on the same 
pixel (not shown), the incident beam of light R is meant to be identified as an incident beam of 
light representative of the pixel. 

[054] An optical center O is defined so that the sum of squares of distances between all 

incident beams of light R and the optical center O can take a minimum value. On the other 



hand, a base point K of an incident beam of light R is defined as a point where a distance d 
between the incident beam of light R representative of a pixel and the optical center O takes a 
minimum value. 

[055] In other words, the optical center O(x 0 , y 0 , z 0 ) is calculated with least-square 

method applied to the sum of squares of distances d relative to the incident beams of light R, 
which are defined by the first and second positions of light sources P\{x { , y { , z { ) and 
P2(x 2 , y 2 , z 2 ). The sum of squares is represented by an expression ( 1 ) . 

di=-(A 2 /B)+C —(1) 

where A, B and C are defined as: 

A = (x 2 - x, X*! - x 0 )+ (y 2 - y x \y x - y 0 )+ (z 2 - z x \z x - z 0 ) 
B = {x 2 -x l ) 2 +(y 2 -y l ) 2 +(z 2 -z l ) 2 
C = (x l - x 0 f + (y, -y 0 f+ {z y - z 0 f 

[056] As described above, the characteristics of a non-pin-hole camera can be quantified by 

introducing the calibration data, which correlates a direction defined by the first and second 
positions PI and P2 of light source for a position of pixel with a displacement from the optical 
center O to the base point K. In this connection, the displacement is represented by a 
three-dimensional vector V D (d x , d y , d\ 

[057] It is understood that other types of calibration data may alternatively be selected. In the 

example described above, the vector V D is defined as a displacement originating from the 
optical center O of a reference point perpendicular to an incident beam of light. It may 
alternatively be possible to adopt any reference point as long as it is spatially fixed relative to 
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the camera. It may also be possible to select any vector for the vector V D as long as it 
originates from the reference point to an arbitrary point on the incident beam of light. 

Method for Generating Calibration Information (table) 

[058] Referring to FIGS.4A and 4B, a method for generating calibration information (table) 

is described, which correlates data produced by quantification of characteristics of a 
non-pin-hole camera with a pixel. FIGS.4A and 4B are schematic diagrams showing the 
principle of generating the calibration information. FIG.4A illustrates conceptually how to 
measure the calibration data for a given incident beam of light by changing pan and tilt of the 
camera. In contrast to this, FIG.4B illustrates a method of measuring the calibration data for a 
fixed camera by changing the incident beam of light. 

[059] As shown in FIG.4A, the calibration information is generated by specifying a direction 

of incident beam of light R imposed on a pixel of the camera unit C in the following steps: 
moving the position of a light source in one direction, one-axis direction defined by the 
positions PI and P2 of light source; determining the incident beam of light R defined by the 
positions PI and P2; and adjusting the pan and tilt of camera unit C in two axes so that the 
beam of light R can impinge on the pixel (measurement pixel). 

[060] As shown in FIG.4B, it may alternatively be possible to specify the direction of 

incident beam of light R which is defined by the positions of PI and P2 of light source 
impinging on the measurement pixel in the following manner. The positions PI and P2 are 
respectively moved in X, Y and Z directions (three-axis directions) so that the incident beam 
of light R can impinge on the measurement pixel, while the camera unit C is fixed. 

[061] Introducing one of the methods described above shown in FIG.4A or FIG. 4B, the 

calibration table is generated from the incident beam of light R specified for the measurement 
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pixel. As shown in FIG.3, the table correlates the direction of incident beam of light R and the 
displacement Vd from the optical center O to the base point K with the measurement pixel. 

Calibration Information Generator 
[062] Referring to FIG.5, the structure of a calibration information generator is described, to 

which a method for generating calibration information according to the present invention is 
applied so as to prepare it. FIG.5 is a perspective view showing the calibration information 
generator. 

[063] A calibration information generator 100 includes a camera support table (rotatable 

stage) 110, which can support a camera unit C and rotate it in pan and tilt directions. The 
generator 100 also includes a three-dimensional table (XYZ stage) 120, which can move a 
point light source L in three-dimensional directions, forward-backward, right-left and 
upward-downward, relative to the camera support table 110. 

[064] The camera support table 1 10 includes a pan table 1 1 1 which is able to rotate about a 

vertical axis on a horizontal plane, and a tilt table 1 12 mounted rotatably to the pan table 111 
about a horizontal axis. The camera unit C is mounted to the tilt table 112. 

[065] The three-dimensional table 120 has an X-axis rail 121 which is laid out in a 

forward-backward direction on a horizontal plane relative to the camera support table 110, and 
a Y-axis rail 122 which is laid out in a right-left direction and able to move in the 
forward-backward direction on the X-axis rail 121, and a Z-axis rail 123 which is laid out in a 
vertical direction and able to move in a right-left direction. The point light source L is 
movably mounted to the Z-axis rail 123 in an upward-downward direction. 

[066] The pan table 111 and tilt table 1 12, which are driven by a driving unit such as a pulse 

motor (not shown), are able to swing the optic axis of camera unit C in upward-downward and 
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right-left directions, respectively. A rotational angle detector such as a rotary encoder (not 
shown), which measures the angle of optic axis for the camera unit C, is attached to a 
rotational shaft for each of the pan table 111 and tilt table 112. 
[067] The Y-axis rail 122, Z-axis rail 123 and point light source L are able to linearly move 

on their corresponding rails on which they are mounted because they are driven by converter 
mechanisms (not shown), which convert the angular movement of pulse motors into linear 
one. 

[068] It is understood that the driving unit and converter mechanism are controlled by a 

control unit (not shown), and the angle of optic axis for camera unit C and the position of point 
light source L arc arranged so that the control unit can access to them. 

Operation of a Calibration Information Generator 
[069] An example of operation of a calibration information generator 100 is described with 

reference to FIG. 6, in addition to FIGS.4A, 4B, and 5. FIG. 6 is a flow chart showing steps for 
generating a calibration table by the calibration information generator 100 with the method 
shown in FIG.4A. 

[070] At a step SI, the calibration information generator 1 00 sets a point light source L at a 

predetermined position, a position PI of light source, and makes rotations for a pan table 111 
and a tilt table 112 about individual two axes so that a beam of light originating from the point 
light source L located at the position PI can impinge on a measurement pixel of a camera unit 
C, thereby obtaining the resultant amounts of pan and tilt. Subsequently at a step S2, the 
generator 100 measures a position relative to the camera unit C for the position PI according 
to the amounts of pan and tilt. 

[071] At a step S3, the generator 100 makes one-axis movement for a Y-axis rail 122 on an 



X-axis rail in a direction of X, forward or backward, thereby shifting the point light source L 

from the position PI to a position P2. 
[072] At a step S4, the generator 100 obtains another set of amounts of pan and tilt for the 

position P2 in the similar manner as that of the step SI. Subsequently at a step S5, the 

generator 100 measures a relative position for the position P2. 
[073] It is possible to specify an incident beam of light R impinging on the measurement 

pixel according to the respective relative positions obtained for the positions PI and P2 at the 

steps S2 and S5, respectively. 
[074] Referring to FIG.7 as well as FIGS.4A, 4B and 5, a method for calculating a position 

of light source relative to the camera unit C is described. FIG.7 is a diagram illustrating a 

position of light source relative to a camera when the camera is panned and tilted. 
[075] It is assumed that the camera unit C is mounted on a fixed reference point B of the 

camera support table 110. If the camera support table 1 10 or the camera unit C is rotated by a 

pan of -9 pan and a tilt of-0 tUt , a direction of light source as viewed from the camera unit C 

will coincides with one which occurs when the light source is rotated about a rotational center 

Ox of the camera support table 110 by 0 pan and 6 m . 
[076] Assuming that coordinates of the position of a light source before rotation are defined 

as P(x l , y 1 , zj, and amounts of pan and tilt e pm and 0 m , respectively, then a position 

P R (R xl , R y i, R zl ) of the light source after rotation can be represented by an expression 

(2). 

cos 6 m cos 6 pan - cos 6 m sin 6 pan sin 6 m Yx^ 

sin0 pan cos0 pan 0 y x —(2) 

- sin 6 m cos 0 pan sin 0 t;lt sin 0 pan cos 6 tilt lz ly 

[077] In this way, it is possible to specify the incident ray of light impinging on a 
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measurement pixel by calculating relative positions for the two positions of the light source 
after the rotation of camera support table 110. Defining coordinates of the position PI 
as(x pl , y pl , z pl ) and P2 as{x p2 , y p2 , z p2 ), which are translated into relative positions, 
the incident beam of light is specified by an expression (3) with a real number of t. 
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[078] Returning to FIG.6, the description of flow is resumed. 

[079] At a step S6, the calibration information generator 100 judges whether or not 

measurement is completed for a predetermined number of measurement pixels. If it is not 
completed (No), the flow returns to the step SI and the generator 100 continues specifying 
another incident beam of light for the next measurement pixel. If it is completed (Yes), the 
generator 100 calculates, at a step S7, an optical center according to the directions of incident 
beams of light which are obtained for the predetermined number of measurement pixels. 

[080] At a step S8, the generator 100 generates a calibration table, which correlates a 

direction of incident beam of light and a displacement from the reference point with each of 
the pixels. It is noted that the generator 100 executes correlation by interpolation according to 
the data of nearby measurement pixels for other pixels which are not subjected to 
measurement. In this connection, it may be preferable to select the positions of measurement 
pixels in advance, taking into account the characteristics of the lens of camera unit C. For 
example, it may be preferable to adopt a position of measurement pixel for every five or less 
pixels in case of a fish-eye lens, or for every five to ten pixels in case of other types of lenses. 
Although the generator 100 described above as an example generates a calibration table, it 
may be possible for the generator 100 to include a predetermined function such as a function 
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for interpolation according to data of nearby measurement pixels. Alternatively, it may be 
possible for the generator 100 to include not only a numerical expression to translate a position 
of pixel into a direction of incident beam of light, but also the other numerical expression to 
translate this position of pixel into its displacement. 
[081] Referring to FIG.8 as well as FIG.5, further description is given to how the camera unit 

C is rotated at the steps SI and S4. FIG.8 is a flow chart showing the steps by which the 
generator 100 directs an incident beam of light so as to impinge on a position of measurement 
pixel. 

[082] As shown in FIG.8, at a step S10, the generator 100 incorporates an image captured by 

the camera unit C. In this connection, it is assumed that only the point light source L is on the 
image, namely a white point on the image against a background of black. 

[083] At a step SI 1, the generator 100 searches for a position (peak position) in which a 

maximum of the light intensity of white point falls. 

[084] Referring to FIG. 9, description is given to how the generator 100 searches for the peak 

position. FIG. 9 is a graph illustrating the relationship between a position of pixel 
P(---, P n _ 2 , P nA , P n , P n+l , P n+2 , •• -)on the horizontal axis and light intensity I on the 
vertical axis. The light intensity I has a distribution with a peak at a certain pixel. 

[085] A peak position P e where the light intensity I takes the maximum value can be 

obtained by numerical expressions (4) and (5). It is defined that I n represents a light intensity 
at a position of pixel P n ; l n _ x and I n+l represent light intensities at positions of pixels P n _ x 
and P n+l next to P n , respectively. 

If h-^h^ 

Pe=P n +(l n *-I n -dlWn-In-d} -(4) 
If I n -l>I n+ l- 



[086] In this connection, searching of the peak position P e is carried out by investigating 

light intensities of neighboring pixels in both vertical and horizontal directions. 

[087] At a step S12, the generator 100 judges if the difference between the peak position P e 

searched for at the step SI 1 and the position of a measurement pixel is less than or equal to a 
predetermined threshold value. If it is (Yes), judging that the peak position P e coincides with 
the position of this measurement pixel, the generator 100 terminates the flow. 

[088] If the difference is greater than the predetermined threshold value (No), at a step S13, 

the generator 100 converts the difference into angles, thereby calculating amounts of rotation 
required for the pan and tilt tables 1 1 1 and 1 12, respectively. At a step SI 4, the generator 100 
rotates the pan and tilt tables 1 1 1 and 1 12 according to the amounts of rotation calculated at 
the step SI 3, and returns the flow to the step S10, repeating the flow until the peak position 
P e coincides with the position of the measurement pixel. 

[089] Although the operation of calibration information generator 100 has been described, 

which generates a calibration table with the method shown in FIG.4A, it may be possible to 
generate it with the other method shown in FIG.4B. 

[090] In this case, the generator 100 determines the position PI of light source by moving the 

position of point light source L in at least one of Y and Z directions, positively or negatively 
so that the maximum light intensity of incident beam of light falls in the measurement pixel 
while the camera unit C is fixed. After changing the distance between the point light source L 
and the camera unit C in an X direction, the generator 100 similarly determines the position P2. 
In this way, determining the positions of PI and P2 of the light source, the generator 100 
specifies the direction of incident beam of light R. After repeating these steps according to the 
number of measurement pixels, the generator 100 conducts the steps S7 and S8, thereby 
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generating a calibration table, 
a. First Embodiment 

[091] Referring to FIG. 10, description is given to a position measurement apparatus. FIG. 10 

is a block diagram illustrating the structure of position measurement apparatus according to 
the present invention. A position measurement apparatus 1 can measure a three-dimensional 
position of an object OB (marker M) according to an image captured by a camera unit C a 
including a set of two cameras CI and C2 (non-pin-hole cameras) , without any mask 
restricting an incident beam of light . The position measurement apparatus 1 includes an image 
input module 10, a pixel position module 20, a storage module 30 and a calculation module 40. 
The marker M, which is for indicating a spot for detection, includes a seal with a given color 
and shape, a light emitting diode of infrared beam and the like. 

[092] The image input module 10 incorporates images of the object OB captured by the 

camera unit C. When the image input module 10 sequentially incorporates dynamic images 
captured by the cameras CI and C2, it synchronizes them. The image input module 10 has a 
memory (not shown) temporarily storing the captured images, to which the pixel position 
module 20 is adapted to have access. 

[093] The pixel position module 20 can detect the position of a pixel, which is representative 

of the object, in each of the images incorporated by the image input module 10. In case of a 
marker M is introduced, the center of gravity for an area containing the image of the marker M 
is considered the position of a pixel that is representative of the marker M. The position of the 
pixel detected for each image is sent to the calculation module 40. 

[094] The storage module 30, which is a storage medium such as a memory, stores a 

calibration table 31 correlating the calibration data with each of the pixels for each of the 
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cameras CI and C2. Calibration tables 31a and 31b are prepared for the cameras CI and C2, 
respectively. The data of calibration table 31 includes the direction of an incident beam of 
light impinging on each of the pixels and the displacement Vd from the reference point (the 
optical center O) to the incident beam of light, as described above with reference to FIG.3. 

[095] As shown in FIG.ll, correlating with a set of coordinates x and y of a pixel, the 

calibration table 3 1 includes the displacement V D [d x , d , d z ) of an incident beam of light 
relative to the optical center O and its direction in a form of angles a and y , which are for 
specifying the incident beam of light impinging on the pixel. 

[096] It may alternatively be possible to adopt a given function or a transformational 

expression instead of the calibration table described above. 

[097] The calculation module 40 can calculate the position (three-dimensional position) of an 

object OB (marker M) according to the data of calibration table 3 1 which is correlated with the 
position of a pixel of an image detected by the pixel position module 20. 

[098] It may be possible to adopt other types of apparatus instead of the position 

measurement apparatus 1 described above. For example, it may be possible for the pixel 
position module 20 to execute matching of images captured by the camera unit C in a 
predetermined size of block so as to detect the position of a pixel representative of an object. 

[099] It may also be possible to use three or more cameras to measure the position of an 

object instead of the camera unit C including a set of two cameras CI and C2. For example, it 
may be possible to adopt a set of nine cameras. In this case, assigning the camera positioned 
centrally as a reference camera, the position of the object can be more accurately measured by 
calculating an average of eight positions detected by combinations of the reference camera and 
the remaining eight cameras. 

[0100] A computer program executes a computer so as to operate its processor and memory 
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including the storage module 30, so that the image input module 10, pixel position module 20 
and calculation module 40, which are included in the position measurement apparatus 1, are 
brought into operation. 

[100] Referring to FIG. 12 in addition to FIG. 10, the operation of a position measurement 

apparatus 1 is described. FIG. 12 is a flow chart illustrating the operation of the apparatus 1. 

[101] At a step S20, an image input module 10 incorporates images captured by a camera 

unit C including two cameras CI and C2. At a step S21, a pixel position module 20 detects a 
position of pixel representative of an object OB (marker M) in each of the images. 

[102] At a step S22, a calculation module 40 reads the calibration data, which is correlated 

with the position of pixel representative of the object OB, from one of calibration tables 31a 
and 31b that are prepared for the cameras CI and C2, respectively. At a step 23, the 
calculation module 40 calculates a three-dimensional position of the object OB according to 
the calibration data. 

[103] Referring to FIG. 13 in addition to FIG. 10, detailed description is given to a method for 

calculating the [[tree]] three -dimensional position, which is conducted at the step S23. FIG. 13 
is a schematic diagram describing the method for calculation. 

[104] As shown in FIG. 13, a compensated optical center is defined as O x {x x , y x , z x ), 

which is compensated according to a displacement derived from the calibration table 31a 
correlated with the position of pixel that is representative of the object OB in the image 
captured by the camera CI. Similarly for the image captured by the camera C2, the other 
compensated optical center is defined as 0 2 (x 2 , y 2 , z 2 ), which is compensated according 
to a displacement derived from the calibration table 31b. 

[105] A direction of incident beam of light, which impinges on the position of pixel for the 

object OB in the image captured by the camera CI, is defined as an azimuth a x and an 
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elevation y x , which are obtained from the calibration table 31a. In the method according to 
this embodiment, the camera captures the image without any mask restricting the incident 
beam of light. Similarly, the other direction of incident beam of light associated with the 
camera C2 is defined as an azimuth a 2 and an elevation y 2 , which are obtained from the 
calibration table 31b. It is noted that the elevation y 2 is not shown in FIG. 13 because it is not 
used for a calculation of position. 
[106] A position of object m[P x , P y , P z ) can be calculated by numerical expressions (6), 

(7) and (8). 

P x = (x l tan^ -y y -x 2 tana 2 +y 2 )/(tana l +tana 2 ) — (6) 

P y = (P x - x 1 )tan a v + y v — (7) 

P z = (P x - x 1 )tan y y + z 1 — (8) 
[107] In this way, the position measurement apparatus 1, which employs non-pin-hole 

cameras, can accurately measure the position of object. 
[108] In this connection, it may be possible to apply a position measurement apparatus 1 

according to the present invention to a movable robot, an automobile and the like. For example, 
introducing the apparatus 1 into the movable robot, it may be possible for the robot to walk 
stably on a floor because it can incorporate information on unevenness of the floor by 
conducting measurement for the position of floor with the apparatus 1 . 

b. Second Embodiment 

[109] A second embodiment of the present invention is described with reference to FIG. 14. 

FIG. 14 is a block diagram illustrating the structure of a position measurement apparatus 
according to the present invention. The same symbol is used for an item which is the same as 
that of the first embodiment and its description would not be repeated. 
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[110] A position measurement apparatus 200 shown in FIG. 14, which has a camera unit C 

capturing an image of a collimated beam of light that originates from a laser unit 250 and 
impinges on an object OB , without any mask restricting the incident beam of light , can 
determine a three-dimensional position of the object OB according to the image. The 
apparatus 200 swings the direction of beam of light upward-downward and right-left so that 
the position of object OB can be determined according to three-dimensional measurement 
conducted at a large number of points. 

[Ill] The apparatus 200 according to the second embodiment preferably includes, the laser 

unit 250, the camera unit C, an image input module 210, a pixel position module 220, a 
storage module 230 and a calculation module 240. 

[112] As shown in FIG. 15, the laser unit 250 includes a laser gun 251, a condenser lens 252 

and a diffraction gating 253. Brief description is given to each item below. The laser gun 251 
produces a laser beam LB, the condenser lens 252 condenses the laser beam LB produced by 
the laser gun 25 1 and the diffraction grating 253 disperses the laser beam LB condensed by the 
condenser lens 252. The diffraction grating 253 disperses the laser beam LB in a vertical 
direction. 

[113] After the diffraction grating 253, a diffusion lens 254 is placed for diffusing the 

dispersed laser beams in one direction so as to form a slit of light. The diffusion lens 254 
includes a cylindrical lens and the like. The diffusion lens 254 diffuses each of the laser beams 
at an angle of 60 deg. 

[114] As shown in FIG. 16, the laser beam LB emitted by the laser unit 250 is diffused in a 

conical shape, reflecting and scattering in a form of a trajectory RL of quadratic curve on a 
plane PL. The reflection and scattering of the laser beam LB indicates a position illuminated 
by the laser beam LB. The laser beam LB is dispersed into five beams, each of which is 
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diffused in an angle of 60 deg. Each of these laser beams is emitted radially from an origin 0 L , 
which is individually defined for each beam. It may be practically preferable to adopt another 
larger number of beams than five described above, which is selected for explanation as an 
example. For example, if an angle B of 32 deg. and an angle C of 1.6 deg. between a pair of 
neighboring beams are selected, the number of beams results in twenty one. 

[115] It may be alternatively possible to adopt a rotation mechanism so as to rotationally 

position a spot laser beam in upward ( x) and lateral (y) directions instead of a lens system for 
diffraction and diffusion used in the laser unit 250 described above. In this case, it may be 
preferable to arrange the mechanism so that the laser beam can be radially emitted from a sole 
point so as to relieve the effort associated with calculation of the position of an object. 

[116] The image input module 210 can incorporate an image of the object OB captured by 

the camera unit C. The image is in a form of color or monochromatic multi-gradation image. 
The image input module 210 has a memory (not shown) which temporarily stores images 
captured by the camera unit C, to which the pixel position module 220 makes access. 

[117] The pixel position module 220 can detects the position illuminated by the laser beam 

LB in the image incorporated by the image input module 210, thereby determining the position 
of pixel. More specifically, judging from the distribution of lightness on the image, the pixel 
position module 220 detects a pixel which has the peak of lightness. The position of pixel thus 
detected for each of the images is sent to the calculation module 240. 

[118] The storage module 230, which is preferably made of a common storage medium such 

as a hard disk, stores a calibration table 231 which correlates calibration data with each of the 
pixels of the image captured by the camera unit C. As shown in FIG.l 1, correlating with a set 
of coordinates x and y of a pixel, the calibration table 231, which is for determining the 
incident beam of light impinging on the pixel, includes a displacement V D (d x , d y , d z ) 



- 25 - 



relative to an optical center 0 and a direction in a form of angles a and y . As shown in 
FIG. 17, the angle a is an azimuth relative to an X-Z plane of Cartesian coordinates 
(X, Y, Z) whose origin is arranged to coincide with the origin 0 L of the laser beam LB 
emitted by the laser unit 250. On the other hand, the angle y is an elevation relative to an 
X-Y plane. 

[119] The calculation module 240 calculates a position (three-dimensional position) of the 

object OB according to the data of calibration table 231, which is correlated with the position 
of pixel of each of the images detected by the pixel position module 220. 

[120] Referring to FIGS. 17 and 18, the operation of a position measurement apparatus 200 is 

described. FIG. 18 is a flow chart showing the steps for operating the apparatus 200. 

[121] At a step ST1, the apparatus 200 illuminates an object OB with a laser beam LB. At a 

step ST2, a camera unit C captures an image of the object OB. At a step ST3, an image input 
module 230 incorporates the image captured at the step ST2. 

[122] At a step ST4, a pixel position module 220 detects a peak of lightness on the image 

which is incorporated by an image input module 210, thereby determining a position of pixel 
which corresponds to the spot illuminated by the laser beam LB on the object OB. 

[123] At a step ST5, accessing to a calibration table 231, a calculation module 240 obtains 

the calibration data correlated with the position of pixel determined at the step ST4. At a step 
ST6, the calculation module 240 calculates a camera base position P C (C X , C , C z ) 
according to the calibration data obtained at the step ST5. 

[124] As shown in FIG. 17, the camera base position Pc is meant to represent an origin from 

which a vector of line-of-sight for the camera unit C starts. If the optical center of camera unit 
C is defined as o(O x , O y , O z ), the camera base position P c is represented by the following 
numerical expressions. 
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C x =O x +d x 
C y =O y +d y 
C z =O z +d z 

[125] In this connection, it may be possible to prepare a calibration table 231 which 

correlates the camera base position P C (C X , C y , C z ) instead of the displacement 
V D (d x , d y , d z ) with a position of pixel. 

[126] At a step ST7, the calculation module 240 calculates a position illuminated by the laser 

beam LB according to the camera base position P C {C X , C , C z ), the angles a and 
y derived from the calibration table 231 in addition to angles co and (j> which define the 
direction of laser beam LB. If the distance of an object is limited, limited laser beams impinge 
on a given area of image. Therefore in this case, it may be possible to determine a laser beam 
according to the position of a pixel, so that the angles co and <j) are accordingly determined. 
If a laser unit is arranged so that it can mechanically swing a spot beam, it may be preferable 
to mechanically or electrically measure the direction to which the spot light is directed. In this 
way, it may be possible to incorporate the angles co and <j> , which are synchronized with the 
image, into the calculation module 240, which calculates a position illuminated by the laser 
beam LB. 

[127] The calculation of a position is conducted as described below. 

[128] As shown in FIG. 17, it is defined that a distance from the origin of the laser beam LB 

to an illumination point P 0 (X, Y, Z) is r and the other distance from the camera base 

position Pc of camera unit C to P 0 (X, Y, z) is R. 
[129] The illumination point P 0 (X, Y, Z) is geometrically related to the direction vector 

of laser beam LB having a distance of r in the following numerical expressions (9), (10) and 

(11). 
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X = rcosco — (9) 

Y = rsinco —(10) 
Z = -rcosa)sin<p — (11) 

[130] The relationship between the illumination point P 0 (X, Y, Z) and the line-of-sight 

vector of camera unit C having a distance of R is also represented by the following numerical 
expressions (12), (13) and (14). 

X = Rcosacosy + C x —(12) 

Y = Rsmacosy + C y —(13) 
Z=-Rsmy + C z —(14) 

The distances r and R can be derived from these expressions (9) through (14). 
For 0<Cy and 0<co, orCy<0 and co < 0 : 

R = (- B + 4(B 2 -AAC))llA 
For 0 = 0: 

R = -C y l$>maco$>y 
For 0<Cyand co<0, or Cy <0 and 0<co : 

R = (-B- 4{B 2 -4AC))/2A 
For o = 0: 

r = [r sin a cos y + C y )/sin co 

Where A, B and C are defined as follows: 

A = cos 2 a cos 2 y + sin 2 y - cos 2 <»sin 2 a cos 2 y/ sin 2 co 

B = C X cos a cos y - C z sin y - C y cos 2 co sin a cosy j sin 2 co 

C = C X 2 +C Z 2 - C 2 cos 2 co /sin 2 co 
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[131] The position measurement apparatus 200 according to the present invention, which 

compensates the non-pin-hole characteristics of the camera unit C, can accurately measure the 
position illuminated by a laser beam. Therefore, it may also be possible to measure a three 
dimensional shape for the object OB by conducting calculation of three-dimensional positions 
for all positions illuminated by the laser beam LB. 

[132] In this connection, it may be possible to apply a position measurement apparatus 200 

according to the present invention to a movable robot, an automobile and the like. The 
apparatus, which is mounted on the movable robot so as to detect a floor, can accurately 
recognize unevenness of the floor, thereby allowing the robot to walk stably. 

[133] Although there have been described what are the present embodiments of the invention, 

it will be understood by persons skilled in the art that variations and modifications may be 
made thereto without departing from the spirit and essence of the invention whose scope is 
indicated by the appended claims. 
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ABSTRACT OF THE DISCLOSURE 

A method for measuring a position of an object according to an image of the object 
captured by a camera unit. The method includes the following steps: calculating a discrepancy 
of an incident beam of light penetrating a lens system of the camera unit relative to an optical 
center of the lens system, and compensating the position of the object according to the 
discrepancy. 
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